Optimal vs. classical linear dimension reduction

We describe a computer intensive method for linear dimension re duction which minimizes the classi cation error directly Simulated annealing Bohachevsky et al is used to solve this problem The classi cation error is determined by an exact integration We avoid distance or scatter measures which are only surrogates to circumvent the classi cation error Simulations in two dimensions and analytical approximations demonstrate the superiority of optimal classi cation opposite to the classical procedures We compare our pro cedure to the well known canonical discriminant analysis homoscedastic case as described in Mc Lachlan and to a method by Young et al for the heteroscedastic case Special emphasis is put on the case when the distance based methods collapse The computer intensive algorithm always achieves minimal classi cation error Introduction Classi cation deals with the allocation of objects to g predetermined groups G f gg say The goal is to minimize the misclassi cation rate over all possible future allocations characterized by the conditional densities pi x i g The minimal error is the so called Bayes error Mc Lachlan Often we want to reduce the dimension of the classi cation problem to one or two dimensions in order to support human imagination without signi cantly increasing the misclassi cation rate This article deals with linear combinations of the original variables to achieve this goal Lin ear Dimension Reduction The next section reviews the classical approach based on distance measures and presents the idea of Young et al in a way that facilitates such a distance formulation Section introduces computerintensive dimension reduction and simulated annealing Section compares the classical and the computerintensive method Classical Linear Dimension Reduction The intuitive idea is to project the data in a way that maximizes the distance between the groups hopefully this will also minimize the misclassi cation rate The distance measure relates the between group scatter matrix